Six reasons for rejecting an industrial survey paper

Context: Despite their importance in any empirically based research program, industrial surveys are not very common in the software engineering literature. In our experience, a possible reason is their difficulty of publication. Goal: We would like to understand what are the issues that may prevent the publication of papers reporting industrial surveys. Method: In this preliminary work, we analyzed the surveys we conducted and extracted the main lessons learned in terms of issues and problems. Results: Most common critics posed to industrial surveys are: lack of novelty, limitation of the geographic scope and sampling issues. Conclusions: Most objections that led to reject a survey paper actually are not easy to overcome and others are not so serious. These objections could restrain researchers from conducting this type of studies that represent an important methodological asset. For these reasons, we think that reviewers should be less severe to judge survey papers provided that all the limitations of the study are well explained and highlighte

Is My Project's Truck Factor Low? Theoretical and Empirical Considerations About the Truck Factor Threshold

The Truck Factor is a simple way, proposed by the agile community, to measure the system's knowledge distribution in a team of developers. It can be used to highlight potential project problems due to the inadequate distribution of the system knowledge. Notwithstanding its relevance, only few studies investigated the Truck Factor and proposed ways to efficiently measure, evaluate and use it. In particular, the effective use of the Truck Factor is limited by the lack of reliable thresholds. In this preliminary paper, we present a theoretical model concerning the Truck Factor and, in particular, we investigate its use to define the maximum achievable Truck Factor value in a project. The relevance of such a value concerns the definition of a reliable threshold for the Truck Factor. Furthermore in the paper, we document an experiment in which we apply the proposed model to real software projects with the aim of comparing the maximum achievable value of the Truck Factor with the unique threshold proposed in literature. The preliminary outcome we achieved shows that the existing threshold has some limitations and problem

Search based path and input data generation for web application testing

Test case generation for web applications aims at ensuring full coverage of the navigation structure. Existing approaches resort to crawling and manual/random input generation, with or without a preliminary construction of the navigation model. However, crawlers might be unable to reach some parts of the web application and random input generation might not receive enough guidance to produce the inputs needed to cover a given path. In this paper, we take advantage of the navigation structure implicitly specified by developers when they write the page objects used for web testing and we define a novel set of genetic operators that support the joint generation of test inputs and feasible navigation paths. On a case study, our tool Subweb was able to achieve higher coverage of the navigation model than crawling based approaches, thanks to its intrinsic ability of generating inputs for feasible paths and of discarding likely infeasible paths

Why Creating Web Page Objects Manually if It Can Be Done Automatically?

Page Object is a design pattern aimed at making web test scripts more readable, robust and maintainable. The effort to manually create the page objects needed for a web application may be substantial and unfortunately existing tools do not help web developers in such task.In this paper we present APOGEN, a tool for the automatic generation of page objects for web applications. Our tool automatically derives a testing model by reverse engineering the target web application and uses a combination of dynamic and static analysis to generate Java page objects for the popular Selenium WebDriver framework. Our preliminary evaluation shows that it is possible to use around 3/4 of the automatic page object methods as they are, while the remaining 1/4 need only minor modifications

Using multi-locators to increase the robustness of web test cases

The main reason for the fragility of web test cases is the inability of web element locators to work correctly when the web page DOM evolves. Web elements locators are used in web test cases to identify all the GUI objects to operate upon and eventually to retrieve web page content that is compared against some oracle in order to decide whether the test case has passed or not. Hence, web element locators play an extremely important role in web testing and when a web element locator gets broken developers have to spend substantial time and effort to repair it. While algorithms exist to produce robust web element locators to be used in web test scripts, no algorithm is perfect and different algorithms are exposed to different fragilities when the software evolves. Based on such observation, we propose a new type of locator, named multi-locator, which selects the best locator among a candidate set of locators produced by different algorithms. Such selection is based on a voting procedure that assigns different voting weights to different locator generation algorithms. Experimental results obtained on six web applications, for which a subsequent release was available, show that the multi-locator is more robust than the single locators (about -30% of broken locators w.r.t. the most robust kind of single locator) and that the execution overhead required by the multiple queries done with different locators is negligible (2-3% at most)

A Method for Developing Model to Text Transformations

In the field of business process development, model transformations play a key role, for example for moving from business process models to either code or inputs for simulation systems, as well as to convert models expressed with notation A into equivalent models expressed with notation B. In the literature, many cases of useful transformations of business process models can be found. However, in general each transformation has been developed in an ad-hoc fashion, at a quite low-level, and its quality is often neglected. To ensure the quality of the transformations is important to apply to them all the well-known software engineering principles and practices, from the requirements definition to the testing activities. For this reason, we propose a method, MeDMoT, for developing non-trivial Model to Text Transformations, which prescribes how to: (1) capture and specify the transformation requirements; (2) design the transformation, (3) implement the transformation and (4) test the transformation. The method has been applied in several case studies, including a transformation of UML business processes into inputs for an agent-based simulator

Challenges of End-to-End Testing with Selenium WebDriver and How to Face Them: A Survey

Proceeding of 2023 IEEE 16th International Conference on Software Testing, Verification and Validation (ICST 2023), 16-20 April 2023, Dublin, Ireland.Modern web applications are complex and used for tasks of primary importance, so their quality must be guaranteed at the highest levels. For this reason, testing techniques (e.g., end-to-end) are required to validate the overall behavior of web applications. One of the most popular tools for testing web applications is Selenium WebDriver. Selenium WebDriver automates the browser to mimic real user actions on the web.While Selenium has made testing easier for many Teams worldwide, it still has its share of challenges. To better understand the challenges and the corresponding solutions adopted we decided to undertake a personal opinion survey from the industry (in total with 78 highly skilled participants) with a focus on the Selenium ecosystem.The results allow understanding which challenges are consid-ered more relevant by professionals in their daily practice and which are the techniques, approaches, and tools they adopt to face them. Therefore, this study is useful to (1) practitioners interested in understanding how to solve the problems they face every day and (2) researchers interested in proposing innovative solutions to problems having a solid industrial impact.This work was partially supported in part by the Ministerio de Ciencia e Innovación-Agencia Estatal de Investigación (10.13039/501100011033) through the H2O Learn project under Grant PID2020-112584RB-C31, and in part by the Madrid Regional Government through the e-Madrid-CM Project under Grant S2018/TCS-4307

Maturity of Software Modelling and Model Driven Engineering: a Survey in the Italian Industry

Background: The main claimed advantage of Model-driven engineering is improvement in productivity. However, few information is available about its actual adoption during software development and maintenance in the industry. Objective: The main aim of this work is investigating the level of maturity in the adoption of software models and of Model-driven engineering in the Italian industry. The perspective is that of software engineering researchers. Method: First, we conducted an exploratory personal opinion survey with 155 Italian software professionals. The data were collected with the help of a web-based on-line questionnaire. Then, we conducted focused interviews with three software professionals to interpret doubtful results. Results: Software modelling is a very relevant phenomenon in the Italian industry. Model-Driven techniques are used in the industry, even if (i) only for a limited extent, (ii) despite a quite generalized dissatisfaction about available tools and (iii) despite a generally low experience of the IT personnel in such techniques. Limitations: Generalization of results is limited due to the sample size. Moreover, possible self-exclusion from participants not interested in modelling could have biased the results. Conclusion: Results reinforce existing evidence regarding the usage of software modelling and (partially of) Model-driven engineering in the industry but highlight several aspects of immaturity of the Italian industry

Enhancing Web Applications Observability through Instrumented Automated Browsers

In software engineering, observability is the ability to determine the current state of a software system based on its external outputs or signals such as metrics, logs, or traces. Web engineers rely on the web browser console as the primary tool to monitor the client-side of web applications during end-to-end tests. However, this is a manual and time-consuming task due to the different browsers available. This paper presents BrowserWatcher, an open-source browser extension providing cross-browser capabilities to observe web applications and automatically gather browser console logs in different browsers (e.g., Chrome, Firefox, or Edge). We have leveraged this extension to conduct an empirical study analyzing the browser console of the top-50 public websites manually and automatically. The results show that BrowserWatcher gathers all the well-known log categories such as console or error traces. It also reveals that each web browser additionally includes other types of logs, which differ among browsers, thus providing distinct pieces of information for the same website.This work was partially supported in part by the Ministerio de Ciencia e Innovación-Agencia Estatal de Investigación, Spain (10.13039/501100011033) through the H2O Learn project under Grant PID2020-112584RB-C31, in part by the Madrid Regional Government through the e-Madrid-CM Project, Spain under Grant S2018/TCS-4307, and in part supported by the Comunidad de Madrid and Universidad Politécnica de Madrid, Spain through the V-PRICIT Research Programme Apoyo a la realización de Proyectos de I+D para jóvenes investigadores UPM-CAM, under Grant APOYOJOVENES-QINIM8-72-PKGQ0J. Funding for Article Processing Charge (APC): Universidad Carlos III de Madrid (Read & Publish Agreement CRUE-CSIC 2023)

Similarity-based Web Element Localization for Robust Test Automation

Non-robust (fragile) test execution is a commonly reported challenge in GUI-based test automation, despite much research and several proposed solutions. A test script needs to be resilient to (minor) changes in the tested application but, at the same time, fail when detecting potential issues that require investigation. Test script fragility is a multi-faceted problem. However, one crucial challenge is how to reliably identify and locate the correct target web elements when the website evolves between releases or otherwise fail and report an issue. This article proposes and evaluates a novel approach called similarity-based web element localization (Similo), which leverages information from multiple web element locator parameters to identify a target element using a weighted similarity score. This experimental study compares Similo to a baseline approach for web element localization. To get an extensive empirical basis, we target 48 of the most popular websites on the Internet in our evaluation. Robustness is considered by counting the number of web elements found in a recent website version compared to how many of these existed in an older version. Results of the experiment show that Similo outperforms the baseline; it failed to locate the correct target web element in 91 out of 801 considered cases (i.e., 11%) compared to 214 failed cases (i.e., 27%) for the baseline approach. The time efficiency of Similo was also considered, where the average time to locate a web element was determined to be 4 milliseconds. However, since the cost of web interactions (e.g., a click) is typically on the order of hundreds of milliseconds, the additional computational demands of Similo can be considered negligible. This study presents evidence that quantifying the similarity between multiple attributes of web elements when trying to locate them, as in our proposed Similo approach, is beneficial. With acceptable efficiency, Similo gives significantly higher effectiveness (i.e., robustness) than the baseline web element localization approach